Predicting Molecular Subtype and Survival of Rhabdomyosarcoma Patients Using Deep Learning of H&E Images: A Report from the Children's Oncology Group.

PURPOSE: Rhabdomyosarcoma (RMS) is an aggressive soft-tissue sarcoma, which primarily occurs in children and young adults. We previously reported specific genomic alterations in RMS, which strongly correlated with survival; however, predicting these mutations or high-risk disease at diagnosis remains a significant challenge. In this study, we utilized convolutional neural networks (CNN) to learn histologic features associated with driver mutations and outcome using hematoxylin and eosin (H&E) images of RMS. EXPERIMENTAL DESIGN: Digital whole slide H&E images were collected from clinically annotated diagnostic tumor samples from 321 patients with RMS enrolled in Children's Oncology Group (COG) trials (1998-2017). Patches were extracted and fed into deep learning CNNs to learn features associated with mutations and relative event-free survival risk. The performance of the trained models was evaluated against independent test sample data (n = 136) or holdout test data. RESULTS: The trained CNN could accurately classify alveolar RMS, a high-risk subtype associated with PAX3/7-FOXO1 fusion genes, with an ROC of 0.85 on an independent test dataset. CNN models trained on mutationally-annotated samples identified tumors with RAS pathway with a ROC of 0.67, and high-risk mutations in MYOD1 or TP53 with a ROC of 0.97 and 0.63, respectively. Remarkably, CNN models were superior in predicting event-free and overall survival compared with current molecular-clinical risk stratification. CONCLUSIONS: This study demonstrates that high-risk features, including those associated with certain mutations, can be readily identified at diagnosis using deep learning. CNNs are a powerful tool for diagnostic and prognostic prediction of rhabdomyosarcoma, which will be tested in prospective COG clinical trials.

A cross-study analysis of drug response prediction in cancer cell lines.

To enable personalized cancer treatment, machine learning models have been developed to predict drug response as a function of tumor and drug features. However, most algorithm development efforts have relied on cross-validation within a single study to assess model accuracy. While an essential first step, cross-validation within a biological data set typically provides an overly optimistic estimate of the prediction performance on independent test sets. To provide a more rigorous assessment of model generalizability between different studies, we use machine learning to analyze five publicly available cell line-based data sets: National Cancer Institute 60, ancer Therapeutics Response Portal (CTRP), Genomics of Drug Sensitivity in Cancer, Cancer Cell Line Encyclopedia and Genentech Cell Line Screening Initiative (gCSI). Based on observed experimental variability across studies, we explore estimates of prediction upper bounds. We report performance results of a variety of machine learning models, with a multitasking deep neural network achieving the best cross-study generalizability. By multiple measures, models trained on CTRP yield the most accurate predictions on the remaining testing data, and gCSI is the most predictable among the cell line data sets included in this study. With these experiments and further simulations on partial data, two lessons emerge: (1) differences in viability assays can limit model generalizability across studies and (2) drug diversity, more than tumor diversity, is crucial for raising model generalizability in preclinical screening.

Dynamic imaging of nascent RNA reveals general principles of transcription dynamics and stochastic splice site selection.

The activities of RNA polymerase and the spliceosome are responsible for the heterogeneity in the abundance and isoform composition of mRNA in human cells. However, the dynamics of these megadalton enzymatic complexes working in concert on endogenous genes have not been described. Here, we establish a quasi-genome-scale platform for observing synthesis and processing kinetics of single nascent RNA molecules in real time. We find that all observed genes show transcriptional bursting. We also observe large kinetic variation in intron removal for single introns in single cells, which is inconsistent with deterministic splice site selection. Transcriptome-wide footprinting of the U2AF complex, nascent RNA profiling, long-read sequencing, and lariat sequencing further reveal widespread stochastic recursive splicing within introns. We propose and validate a unified theoretical model to explain the general features of transcription and pervasive stochastic splice site selection.

A Deep Learning Pipeline for Nucleus Segmentation.

Deep learning is rapidly becoming the technique of choice for automated segmentation of nuclei in biological image analysis workflows. In order to evaluate the feasibility of training nuclear segmentation models on small, custom annotated image datasets that have been augmented, we have designed a computational pipeline to systematically compare different nuclear segmentation model architectures and model training strategies. Using this approach, we demonstrate that transfer learning and tuning of training parameters, such as the composition, size, and preprocessing of the training image dataset, can lead to robust nuclear segmentation models, which match, and often exceed, the performance of existing, off-the-shelf deep learning models pretrained on large image datasets. We envision a practical scenario where deep learning nuclear segmentation models trained in this way can be shared across a laboratory, facility, or institution, and continuously improved by training them on progressively larger and varied image datasets. Our work provides computational tools and a practical framework for deep learning-based biological image segmentation using small annotated image datasets. Published [2020]. This article is a U.S. Government work and is in the public domain in the USA.

AI Meets Exascale Computing: Advancing Cancer Research With Large-Scale High Performance Computing.

The application of data science in cancer research has been boosted by major advances in three primary areas: (1) Data: diversity, amount, and availability of biomedical data; (2) Advances in Artificial Intelligence (AI) and Machine Learning (ML) algorithms that enable learning from complex, large-scale data; and (3) Advances in computer architectures allowing unprecedented acceleration of simulation and machine learning algorithms. These advances help build in silico ML models that can provide transformative insights from data including: molecular dynamics simulations, next-generation sequencing, omics, imaging, and unstructured clinical text documents. Unique challenges persist, however, in building ML models related to cancer, including: (1) access, sharing, labeling, and integration of multimodal and multi-institutional data across different cancer types; (2) developing AI models for cancer research capable of scaling on next generation high performance computers; and (3) assessing robustness and reliability in the AI models. In this paper, we review the National Cancer Institute (NCI) -Department of Energy (DOE) collaboration, Joint Design of Advanced Computing Solutions for Cancer (JDACS4C), a multi-institution collaborative effort focused on advancing computing and data technologies to accelerate cancer research on three levels: molecular, cellular, and population. This collaboration integrates various types of generated data, pre-exascale compute resources, and advances in ML models to increase understanding of basic cancer biology, identify promising new treatment options, predict outcomes, and eventually prescribe specialized treatments for patients with cancer.

Transcriptional Bursting and Co-bursting Regulation by Steroid Hormone Release Pattern and Transcription Factor Mobility.

Genes are transcribed in a discontinuous pattern referred to as RNA bursting, but the mechanisms regulating this process are unclear. Although many physiological signals, including glucocorticoid hormones, are pulsatile, the effects of transient stimulation on bursting are unknown. Here we characterize RNA synthesis from single-copy glucocorticoid receptor (GR)-regulated transcription sites (TSs) under pulsed (ultradian) and constant hormone stimulation. In contrast to constant stimulation, pulsed stimulation induces restricted bursting centered around the hormonal pulse. Moreover, we demonstrate that transcription factor (TF) nuclear mobility determines burst duration, whereas its bound fraction determines burst frequency. Using 3D tracking of TSs, we directly correlate TF binding and RNA synthesis at a specific promoter. Finally, we uncover a striking co-bursting pattern between TSs located at proximal and distal positions in the nucleus. Together, our data reveal a dynamic interplay between TF mobility and RNA bursting that is responsive to stimuli strength, type, modality, and duration.

Topological Structure Determination of RNA Using Small-Angle X-Ray Scattering.

Knowledge of RNA three-dimensional topological structures provides important insight into the relationship between RNA structural components and function. It is often likely that near-complete sets of biochemical and biophysical data containing structural restraints are not available, but one still wants to obtain knowledge about approximate topological folding of RNA. In this regard, general methods for determining such topological structures with minimum readily available restraints are lacking. Naked RNAs are difficult to crystallize and NMR spectroscopy is generally limited to small RNA fragments. By nature, sequence determines structure and all interactions that drive folding are self-contained within sequence. Nevertheless, there is little apparent correlation between primary sequences and three-dimensional folding unless supplemented with experimental or phylogenetic data. Thus, there is an acute need for a robust high-throughput method that can rapidly determine topological structures of RNAs guided by some experimental data. We present here a novel method (RS3D) that can assimilate the RNA secondary structure information, small-angle X-ray scattering data, and any readily available tertiary contact information to determine the topological fold of RNA. Conformations are firstly sampled at glob level where each glob represents a nucleotide. Best-ranked glob models can be further refined against solvent accessibility data, if available, and then converted to explicit all-atom coordinates for refinement against SAXS data using the Xplor-NIH program. RS3D is widely applicable to a variety of RNA folding architectures currently present in the structure database. Furthermore, we demonstrate applicability and feasibility of the program to derive low-resolution topological structures of relatively large multi-domain RNAs.

Laparoscopic stereoscopic augmented reality: toward a clinically viable electromagnetic tracking solution.

The purpose of this work was to develop a clinically viable laparoscopic augmented reality (AR) system employing stereoscopic (3-D) vision, laparoscopic ultrasound (LUS), and electromagnetic (EM) tracking to achieve image registration. We investigated clinically feasible solutions to mount the EM sensors on the 3-D laparoscope and the LUS probe. This led to a solution of integrating an externally attached EM sensor near the imaging tip of the LUS probe, only slightly increasing the overall diameter of the probe. Likewise, a solution for mounting an EM sensor on the handle of the 3-D laparoscope was proposed. The spatial image-to-video registration accuracy of the AR system was measured to be [Formula: see text] and [Formula: see text] for the left- and right-eye channels, respectively. The AR system contributed 58-ms latency to stereoscopic visualization. We further performed an animal experiment to demonstrate the use of the system as a visualization approach for laparoscopic procedures. In conclusion, we have developed an integrated, compact, and EM tracking-based stereoscopic AR visualization system, which has the potential for clinical use. The system has been demonstrated to achieve clinically acceptable accuracy and latency. This work is a critical step toward clinical translation of AR visualization for laparoscopic procedures.

Software-based PET-MR image coregistration: combined PET-MRI for the rest of us!

BACKGROUND: With the introduction of hybrid positron emission tomography/magnetic resonance imaging (PET/MRI), a new imaging option to acquire multimodality images with complementary anatomical and functional information has become available. Compared with hybrid PET/computed tomography (CT), hybrid PET/MRI is capable of providing superior anatomical detail while removing the radiation exposure associated with CT. The early adoption of hybrid PET/MRI, however, has been limited. OBJECTIVE: To provide a viable alternative to the hybrid PET/MRI hardware by validating a software-based solution for PET-MR image coregistration. MATERIALS AND METHODS: A fully automated, graphics processing unit-accelerated 3-D deformable image registration technique was used to align PET (acquired as PET/CT) and MR image pairs of 17 patients (age range: 10 months-21 years, mean: 10 years) who underwent PET/CT and body MRI (chest, abdomen or pelvis), which were performed within a 28-day (mean: 10.5 days) interval. MRI data for most of these cases included single-station post-contrast axial T1-weighted images. Following registration, maximum standardized uptake value (SUVmax) values observed in coregistered PET (cPET) and the original PET were compared for 82 volumes of interest. In addition, we calculated the target registration error as a measure of the quality of image coregistration, and evaluated the algorithm's performance in the context of interexpert variability. RESULTS: The coregistration execution time averaged 97±45 s. The overall relative SUVmax difference was 7% between cPET-MRI and PET/CT. The average target registration error was 10.7±6.6 mm, which compared favorably with the typical voxel size (diagonal distance) of 8.0 mm (typical resolution: 0.66 mm × 0.66 mm × 8 mm) for MRI and 6.1 mm (typical resolution: 3.65 mm × 3.65 mm × 3.27 mm) for PET. The variability in landmark identification did not show statistically significant differences between the algorithm and a typical expert. CONCLUSION: We have presented a software-based solution that achieves the many benefits of hybrid PET/MRI scanners without actually needing one. The method proved to be accurate and potentially clinically useful.

On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization.

PURPOSE: Common camera calibration methods employed in current laparoscopic augmented reality systems require the acquisition of multiple images of an entire checkerboard pattern from various poses. This lengthy procedure prevents performing laparoscope calibration in the operating room (OR). The purpose of this work was to develop a fast calibration method for electromagnetically (EM) tracked laparoscopes, such that the calibration can be performed in the OR on demand. METHODS: We designed a mechanical tracking mount to uniquely and snugly position an EM sensor to an appropriate location on a conventional laparoscope. A tool named fCalib was developed to calibrate intrinsic camera parameters, distortion coefficients, and extrinsic parameters (transformation between the scope lens coordinate system and the EM sensor coordinate system) using a single image that shows an arbitrary portion of a special target pattern. For quick evaluation of calibration results in the OR, we integrated a tube phantom with fCalib prototype and overlaid a virtual representation of the tube on the live video scene. RESULTS: We compared spatial target registration error between the common OpenCV method and the fCalib method in a laboratory setting. In addition, we compared the calibration re-projection error between the EM tracking-based fCalib and the optical tracking-based fCalib in a clinical setting. Our results suggest that the proposed method is comparable to the OpenCV method. However, changing the environment, e.g., inserting or removing surgical tools, might affect re-projection accuracy for the EM tracking-based approach. Computational time of the fCalib method averaged 14.0 s (range 3.5 s-22.7 s). CONCLUSIONS: We developed and validated a prototype for fast calibration and evaluation of EM tracked conventional (forward viewing) laparoscopes. The calibration method achieved acceptable accuracy and was relatively fast and easy to be performed in the OR on demand.

The Utility of Cloud Computing in Analyzing GPU-Accelerated Deformable Image Registration of CT and CBCT Images in Head and Neck Cancer Radiation Therapy.

The images generated during radiation oncology treatments provide a valuable resource to conduct analysis for personalized therapy, outcomes prediction, and treatment margin optimization. Deformable image registration (DIR) is an essential tool in analyzing these images. We are enhancing and examining DIR with the contributions of this paper: 1) implementing and investigating a cloud and graphic processing unit (GPU) accelerated DIR solution and 2) assessing the accuracy and flexibility of that solution on planning computed tomography (CT) with cone-beam CT (CBCT). Registering planning CTs and CBCTs aids in monitoring tumors, tracking body changes, and assuring that the treatment is executed as planned. This provides significant information not only on the level of a single patient, but also for an oncology department. However, traditional methods for DIR are usually time-consuming, and manual intervention is sometimes required even for a single registration. In this paper, we present a cloud-based solution in order to increase the data analysis throughput, so that treatment tracking results may be delivered at the time of care. We assess our solution in terms of accuracy and flexibility compared with a commercial tool registering CT with CBCT. The latency of a previously reported mutual information-based DIR algorithm was improved with GPUs for a single registration. This registration consists of rigid registration followed by volume subdivision-based nonrigid registration. In this paper, the throughput of the system was accelerated on the cloud for hundreds of data analysis pairs. Nine clinical cases of head and neck cancer patients were utilized to quantitatively evaluate the accuracy and throughput. Target registration error (TRE) and structural similarity index were utilized as evaluation metrics for registration accuracy. The total computation time consisting of preprocessing the data, running the registration, and analyzing the results was used to evaluate the system throughput. Evaluation showed that the average TRE for GPU-accelerated DIR for each of the nine patients was from 1.99 to 3.39 mm, which is lower than the voxel dimension. The total processing time for 282 pairs on an Amazon Web Services cloud consisting of 20 GPU enabled nodes took less than an hour. Beyond the original registration, the cloud resources also included automatic registration quality checks with minimal impact to timing. Clinical data were utilized in quantitative evaluations, and the results showed that the presented method holds great potential for many high-impact clinical applications in radiation oncology, including adaptive radio therapy, patient outcomes prediction, and treatment margin optimization.

Graphics Processing Unit-Accelerated Nonrigid Registration of MR Images to CT Images During CT-Guided Percutaneous Liver Tumor Ablations.

RATIONALE AND OBJECTIVES: Accuracy and speed are essential for the intraprocedural nonrigid magnetic resonance (MR) to computed tomography (CT) image registration in the assessment of tumor margins during CT-guided liver tumor ablations. Although both accuracy and speed can be improved by limiting the registration to a region of interest (ROI), manual contouring of the ROI prolongs the registration process substantially. To achieve accurate and fast registration without the use of an ROI, we combined a nonrigid registration technique on the basis of volume subdivision with hardware acceleration using a graphics processing unit (GPU). We compared the registration accuracy and processing time of GPU-accelerated volume subdivision-based nonrigid registration technique to the conventional nonrigid B-spline registration technique. MATERIALS AND METHODS: Fourteen image data sets of preprocedural MR and intraprocedural CT images for percutaneous CT-guided liver tumor ablations were obtained. Each set of images was registered using the GPU-accelerated volume subdivision technique and the B-spline technique. Manual contouring of ROI was used only for the B-spline technique. Registration accuracies (Dice similarity coefficient [DSC] and 95% Hausdorff distance [HD]) and total processing time including contouring of ROIs and computation were compared using a paired Student t test. RESULTS: Accuracies of the GPU-accelerated registrations and B-spline registrations, respectively, were 88.3 ± 3.7% versus 89.3 ± 4.9% (P = .41) for DSC and 13.1 ± 5.2 versus 11.4 ± 6.3 mm (P = .15) for HD. Total processing time of the GPU-accelerated registration and B-spline registration techniques was 88 ± 14 versus 557 ± 116 seconds (P < .000000002), respectively; there was no significant difference in computation time despite the difference in the complexity of the algorithms (P = .71). CONCLUSIONS: The GPU-accelerated volume subdivision technique was as accurate as the B-spline technique and required significantly less processing time. The GPU-accelerated volume subdivision technique may enable the implementation of nonrigid registration into routine clinical practice.